Repository navigation
feat: add together to native OpenAI-compatible provider routing - #8804
Felix Shi (shi-felix) wants to merge 3 commits into
Conversation
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
Strix Security ReviewWarning This pull request has 117 commits after the last Strix review ( ✅ No open security findings. PR summary: Adds `together` as a native OpenAI-compatible provider so `LLM(model="together/<org>/<model>")` routes through `OpenAICompatibleCompletion` instead of falling back to LiteLLM, wired up via a new `ProviderConfig` entry and matching routing entries, with tests and localized docs. Review summaryReviewed the full diff: the new Ruled out (2)
Review 1 on this PR · Reviewed by Strix |
Summary
togethernow routes through the nativeOpenAICompatibleCompletioninstead of falling back to LiteLLM, soLLM(model="together/<organization>/<model-id>")works without the LiteLLM extra.Fixes #8011.
Changes
lib/crewai/src/crewai/llms/providers/openai_compatible/completion.py: add thetogetherProviderConfig(https://api.together.ai/v1,TOGETHER_API_KEY, optionalTOGETHER_BASE_URL) and list it in the class docstring.lib/crewai/src/crewai/llm.py: addtogethertoSUPPORTED_NATIVE_PROVIDERS,PROVIDER_ALIASES, the native-provider set in_get_native_provider, and_matches_provider_pattern.lib/crewai/tests/llms/openai_compatible/test_openai_compatible.py: coverage for the registry entry, the missing-key error (and that an OpenAI key is not reused), prefixed and explicit-provider routing,TOGETHER_BASE_URL, explicit-config precedence, org/model ids, and the resolved client params.docs/edge/{en,ar,ko,pt-BR}/concepts/llms.mdx: a Together AI accordion.Only the
together/prefix is stripped, sotogether/Qwen/Qwen3-235B-A22B-Instruct-2507-tputreaches the API asQwen/Qwen3-235B-A22B-Instruct-2507-tput. No model catalog is hard-coded.is_litellm=Truestill forces the LiteLLM path, and the other native providers are untouched.Testing
Python 3.13.14 on Windows. No provider credentials and no paid API calls were used.
AI assistance
Written with AI assistance (WorkBuddy) and reviewed before submission.
Note
Low Risk
Scoped additive change to LLM routing and OpenAI-compatible provider config, with tests and docs; existing providers and LiteLLM fallback behavior are unchanged unless
is_litellm=True.Overview
Adds Together AI as a first-class native provider so
LLM(model="together/…")usesOpenAICompatibleCompletioninstead of LiteLLM (nocrewai[litellm]extra).Configuration follows the other OpenAI-compatible hosts: default base
https://api.together.ai/v1, requiredTOGETHER_API_KEY, optionalTOGETHER_BASE_URLor explicitapi_key/base_url. Auth is provider-specific—anOPENAI_API_KEYalone does not satisfy Together.Model strings keep the Together
organization/modelpath after routing; only thetogether/prefix is removed (e.g.together/Qwen/…→Qwen/…at the API). Routing is wired throughllm.pyprovider lists and pattern matching, with tests for registry, routing, env overrides, and client params.Docs add a Together AI accordion in en, ar, ko, and pt-BR LLM guides, aligned with the DeepInfra section.
Reviewed by Cursor Bugbot for commit d9d8e07. Bugbot is set up for automated code reviews on this repo. Configure here.